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Summary of Review 

The Cato Institute report examines international evidence on outcomes from public and 
private education. The paper makes three key claims: private schools outperform public 
schools in “the overwhelming majority of cases”; private schools’ superiority is greatest in 
countries where the education system has more market features; and “the implications for 
U.S. education policy are profound.” Each claim is problematic. The first is based on an 
atypical method of summarizing academic literature and excludes two important research 
studies. The claim also fails to adequately take into account selection bias due, for instance, 
to parents choosing private schools because of an academic focus on their children. The 
second claim oversimplifies a very complex issue, namely the optimal application of mar- 
ket forces to improve education. And the third claim is dubious as well: even if the report’s 
first two claims are legitimate (based on international evidence), there may be no practical 
implications for U.S. education policy. 
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Review 



I. Introduction 

The introduction of market reforms to edu- 
cation systems has been a popular policy 
over the last decade. Charter schools, 
voucher programs, increased home- 
schooling, and tax credits have all intro- 
duced greater competition and market forces 
into the American school system. At issue is 
whether these reforms have improved the 
quality of education. 

Andrew Coulson’s policy analysis report, 
Markets vs. Monopolies in Education. A 
Global Review of the Evidence , endeavors 
to consider all the evidence on this impor- 
tant issue, domestic as well as international. 
The report attempts to answer a provocative 
question: “Would families and communities 
be better served by a free and competitive 
education marketplace than they are by our 
current system of state school monopo- 
lies?” 2 

The author claims that the evidence typically 
used to address this question is “inadequate 
or even irrelevant.” Instead, he advocates for 
the use of international evidence. The rea- 
soning is this: if there is a large volume of 
international evidence and it all points to the 
same conclusion, then that conclusion 
should also apply to the U.S. 

The report collects evidence from 55 domes- 
tic and international studies from over 20 
countries on public versus private school 
performance. Eight dimensions of perform- 
ance are covered: achievement, efficiency, 
parental satisfaction, classroom orderliness, 
condition of facilities, subsequent earnings, 
attainment (graduation rates of high schools, 
or highest average grade completed), and 
intelligence. Studies are then classified ac- 



cording to the freedom of the nation’s edu- 
cational system. The three criteria for a rela- 
tively free education system are: parents pay 
one-third or more of the cost of private 
school, private schools have managerial 
autonomy; and public schools receive at 
least 30 percent more government funding 
per pupil than most private schools. 

This evidence is then used to answer two 
questions: Across the globe, do private 
schools outperform public schools? and Is 
the private school advantage even greater in 
freer education systems? If the answers to 
both questions are “yes,” then the report 
offers the admonition that policymakers 
should introduce as much market reform as 
possible. 

II. Findings and Conclusions 
Of The report 

Using an approach that tallies positive and 
negative conclusions, the report finds that 
the literature strongly supports the conclu- 
sion that private schools outperform public 
ones. Across all eight dimensions of per- 
formance, the number of studies finding an 
advantage for private schools exceeds the 
number finding the opposite. For example, 
of the 63 separate tests for achievement dif- 
ferences, 41 find a statistically significant 
private school advantage; only eight find an 
advantage in favor of public schools (with 
14 reporting no difference). No study finds 
public schools are better in terms of parental 
satisfaction, quality of facilities, or attain- 
ment. Based on a simple count of all 113 
findings, nearly eight times as many favor 
private over public schools. 

To identify public/private differences in 
systems with greater freedom, the report 
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selects 26 of the 55 studies. These are se- 
lected if they meet all three criteria of a free 
system. In this subset of studies private 
schools are found to outperform public 
schools on all dimensions. For example, in 
comparisons of achievement, 15 studies 
favor private schools and only two favor 
public schools. Applying a simple count of 
all 38 tests in these 26 studies, 35 favor pri- 
vate schools compared with two that show a 
public school advantage. 

The report draws on these tabulations of 
evidence to make several broad policy 
claims (page 11): 

• The content of schooling does not need 
to be overseen by the state. 

• There should be universal access to 
minimally regulated education markets. 

• Parents should directly pay at least some 
of the cost of their children’s education. 

III. The Report’s Rationale for its 
Findings and Conclusions 

The report’s own rationale for its findings is 
uncomplicated and based on three asser- 
tions. All the research evidence has been 
collected, correctly classified, and appropri- 
ately summarized. Overwhelmingly, private 
schools appear to outperform public ones. 
Moreover, given the magnitude of the dif- 
ference in performance, no alternative way 
of summarizing the evidence is likely to 
overturn this result. 

As discussed below, however, this rationale 
is very much open to question. 

IV. The Report’s Use of 
Research Literature 

Although the report does include a substan- 
tial body of literature, its use of that litera- 
ture is problematic on several grounds. 



First, although the stated goal of the report is 
to be comprehensive, it omits some relevant 
research. Surprisingly, some of that research 
might support the claims of the report. A 
series of research studies in England, for 
instance, examined how market forces might 
improve its “monopoly” system of educa- 
tion. In general, that research found market 
reforms to be beneficial. 3 

However, other research omitted from this 
review does not support the report’s claims. 
These omitted studies raise serious questions 
about the report’s methodological assump- 
tions and about the usefulness of reviewing 
international evidence instead of relying on 
U.S. research. 4 

Specifically, in their review across ten Latin 
American countries, Somers et al. affirm 
that many earlier studies comparing public 
and private schools have failed to properly 
control for correlated characteristics. 5 As an 
illustration, imagine a study comparing the 
performance of private school students to 
students at a nearby urban public school. A 
researcher should control for differences in 
family income across the schools; any ob- 
served test score difference may arise be- 
cause of family income and not the quality 
of school. There are many such factors to 
control for, including family background, 
student ability, neighborhood resources. 
Based on more complete econometric mod- 
els, Somers et al. find that: 

conditioning on a complete set of 
student, family and peer characteris- 
tics explains a large portion of the 
observed difference in achievement 
between public and private schools... 
Across the 10 countries ... the mean 
private school effect is approxi- 
mately zero. 6 

This omitted research study is significant for 



http://epicpolicv.org/thinktank/review-markets-vs-monopolies 



Page 2 of 9 



two reasons. First, it controls for characteris- 
tics that the new Cato report states are 
unlikely to influence the results, and it finds 
they do matter. Second, the 10 countries are 
all in Latin America, which is represented 
heavily in the report’s literature summary. 

Another study not included casts more doubt 
on the report’s conclusions insofar as they 
apply to the U.S. school system. In a paper 
released in August 2008 (and therefore 
probably too late to be found by Mr. Coul- 
son’s search), which will be published in the 
Annual Review of Economics, Cecilia Rouse 
and Lisa Barrow review the evidence on 
U.S. voucher programs. They conclude: 

The best research to date finds rela- 
tively small achievement gains for 
students offered education vouchers, 
most of which are not statistically 
different from zero. Further, what lit- 
tle evidence exists regarding the po- 
tential for public schools to respond 
to increased competitive pressure 
generated by vouchers suggests that 
one should remain wary that large 
improvements would result from a 
more comprehensive voucher sys- 
tem. 7 

This conclusion of Rouse and Barrow rests 
on a very sophisticated and detailed interpre- 
tation of all the U.S. evidence. 

The very modest conclusions of these two, 
more careful studies are in sharp contrast to 
those in this report. As discussed below, 
simply tallying results of unscreened studies 
is not a particularly useful way of summariz- 
ing research. 

A second way in which the literature review 
is problematic is that it relies heavily on 
unpublished research. Of the 55 research 
papers cited in the literature summary 32 are 



working papers or conference papers. Of 
those that are formally published, not all 
were published in peer-reviewed journals. 
Typically, literature reviews give more cre- 
dence to published studies as one indicator 
of quality, while referring to unpublished 
studies in a supplementary way. This report 
makes no distinction. 

A third concern is that the included studies 
are not representative of the globe in any 
meaningful sense. Of the 55 studies, 23 refer 
to the market reforms in Chile. Another 16 
studies refer to the U.S. education system 
and a full five of those U.S. studies test for 
public/private differences in one city: Mil- 
waukee, Wisconsin. 

Accordingly, almost half the report’s inter- 
national evidence for a private school advan- 
tage is based on studies of the Chilean 
school system. Certainly, many economists 
and other researchers consider the Chilean 
experience to be the most expansive market 
reform of education; it is also the one that 
has received the most intensive research 
investigation (hence its prominence here). 
The research on Chile is far from conclusive 
in identifying private schools’ superiority, 

o 

however. Another surprising finding with 
regard to Chile is its very weak relative 
standing on international tests, despite its 
considerable support for private schools. 9 

When the sample for analysis is reduced to 
include only those studies in systems with 
greater freedom, only a few countries are 
covered. Of the 26 studies, six refer to India 
and four refer to Tanzania. Another five 
refer to the U.S. Interestingly, Chile is not 
represented at all in the reduced sample. 

Finally, even the cited research literature 
does not fully correspond with the author’s 
argument. The report claims that “existing 
[U.S. school choice] programs are too small, 



http://epicpolicv.org/thinktank/review-markets-vs-monopolies 



Page 3 of 9 



too restriction laden, or both” (executive 
summary) and so we should survey educa- 
tion systems across the globe. But a number 
of the studies included in this review are of 
such small programs; they are not system- 
wide comparisons. To take the U.S. exam- 
ples: enrollments were 19,000 in the Mil- 
waukee Parental Choice Program (2007-08); 
5,700 in the Cleveland Scholarship and Tu- 
toring Program (2004); 2,791 in the first 
year of the three-city voucher experiment 
reported in Howell and Peterson (2002); and 
1,400 in the DC Opportunity Scholarship 
program (2004). It is not clear why these 
studies of small-scale programs should be 
included in a review that purports to summa- 
rize international evidence across school 
systems. 

V. Review of the Report’s 
Methods 

A virtue of this report is that the method 
used is easy to understand. A simple distinc- 
tion between public (monopoly) and private 
schools is set up. The evidence is tabulated 
and counted. Public and private schools are 
judged based on which have the most stud- 
ies in their favor. 

Unfortunately, the debate is more complex 
than this. Consequently, more detailed 
analysis is also required. 

The report employs a rhetorical device to 
simplify what is actually a very complex 
issue. The public school system is stated to 
be a “monopoly,” but the author never speci- 
fies which features make it so (both abso- 
lutely and relative to other countries). The 
standard textbook definition of a monopoly 
rests on “barriers to entry” exemplified by 
families who want different schools from the 
current offering(s) but are not allowed them. 
While we shouldn’t minimize real eco- 
nomic, informational and logistic obstacles 



to school choice options in the U.S., there 
remains plenty of opportunity for American 
families to choose different schools: they 
can enroll in charter schools; they can home- 
school; they can move to a different school 
district; or they can pay extra for private 
school. Certainly, private schools compete at 
a disadvantage to subsidized public schools, 
but within the public system there is consid- 
erable heterogeneity. Moreover, other ele- 
ments of “monopoly” exist elsewhere but 
not in the U.S.: countries such as the United 
Kingdom and France have a national cur- 
riculum that all schools — including private 
ones — must follow; similarly, the Nether- 
lands has public and privately owned 
schools, but both are heavily regulated. In 
short, the monopoly features of the U.S. 
education, relative to other countries, are far 
from clear. 

Similarly, the “market” concept used in the 
report is not straightforward. As the author 
notes, citing Merrifield, the market system 
includes “profit, price change, market entry, 
and product differentiation” (p.2). But none 
of these elements are addressed here. There 
is, for instance, no investigation of the role 
of profit in education. In fact, most private 
schools, inside as well as outside the U.S., 
are non-profit enterprises. There is no treat- 
ment of prices, which might allow for cheap, 
low-quality private schools as well as ex- 
pensive, high-quality ones. Finally, product 
differentiation is presumed away; in this 
report, schools are compared by how their 
students perform according to a common 
metric. 

In summary, when commentators style the 
public school system as a monopoly, it is 
important to ask: In what sense? Without 
specifics, it is not clear what to test for to 
see whether private schools are better. For 
example, do private schools have more effi- 
cient managers, motivated by profit? Do 
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they have more flexibility over the curricu- 
lum or staffing? If private schools in the 
Netherlands offer an exemplar, might that 
mean that U.S. private schools should be 
more highly regulated rather than, as this 
Cato report argues, that the U.S. should 
move toward further deregulation? 

A second way in which the method is overly 
simple is in how the evidence is weighed. 
The report’s method for appraising evidence 
is essentially a “vote count” of all studies. 
Each study is given a value of 1 if it finds 
private schools outperform public schools, 0 
if no difference is found, and -1 if public 
schools are superior. These scores are then 
tabulated to see which type of school is su- 
perior. A second tabulation is then per- 
formed for the reduced sample, restricted to 
studies that meet all three market criteria 
and so have the most “market-like” features. 
Again, these studies are counted and tabu- 
lated as to which school sector is superior. 

The problem with using vote count methods 
is that not all votes — not all studies — are 
necessarily equal. There are two ways in 
which these “votes” are not equal. 

First, some studies are better able to detect 
genuine differences in school quality than 
others. For example, it does not seem rea- 
sonable to equate a study based on random 
assignment of students to public and private 
schools with a correlational study that sim- 
ply compares student outcomes subsequent 
to their independent choices of public or 
private school. The latter could easily con- 
found preexisting student differences with 
differences caused by school quality. 

It is possible for a random assignment study 
to be poorly implemented (or have low ex- 
ternal validity). But scholars generally agree 
that there is a hierarchy of methods that al- 
low for causal claims about outcome meas- 



ures, with experimental methods at the top. 
Similarly, as noted above, some studies are 
published only after a rigorous peer review, 
while others are self-published or published 
in a non-peer-reviewed journal. The vote 
count approach assumes that all studies are 
equally valid and useful. 

Second, no account is taken of the power of 
each finding. For example, if two studies 
find a small positive impact from private 
schooling and one finds a large negative 
impact, the vote count method would find 
unambiguously in favor of private schools; 
averaging the three studies would possibly 
reverse the conclusion. Also, one study may 
be based on a small sample and another on 
population-wide data. Although this report 
catalogs sufficient studies in favor of private 
schools that this possibility is unlikely, the 
vote count procedure is still uninformative 
as to the size of any advantage from attend- 
ing a private school. It merely shows (setting 
aside the other problems identified in this 
review) that private schools are probably 
better than public schools. 

But even this conclusion is premature. Im- 
portantly, it is far from clear that any (or all) 
of these studies have properly identified a 
private school effect. Fet us assume that 
private school students outperform public 
school students on raw achievement tests. 
This advantage may be because private 
school students are from wealthier families, 
so studies must control for this. Most do, 
although some don’t. More importantly, 
there may be other hard-to-observe charac- 
teristics that cause a private school advan- 
tage, such as parental engagement, the safety 
of the neighborhood or family religiosity. 10 
Studies do not typically control for all these 
factors and often cannot. Moreover, it is 
necessary to control for the decision to 
choose a private school. That is, families 
choose private school because it is a better 
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fit for them for many reasons; public school 
parents may or may not be doing the same. 
This selectivity bias may be substantial. 

The report explicitly states that selectivity 
bias is not a concern. The author contends 
that either the bias is unimportant, or enough 
of the studies do control for it, or in fact the 
bias is in favor of public schools. But each 
of these reasons is questionable. It is likely 
that the choice of school is motivated by the 
family’s expectation about how well the 
child will do. Simply put, these studies do 
not all include the controls that they should. 
As noted above, Somers et al. illustrate how 
studies that fail to control for peer group 
characteristics can overstate the private 
school advantage. 11 Finally, the report never 
explains why families who enroll in private 
school might have characteristics associated 
with lower achievement. 

VI. Review of the Validity of the 
Findings And Conclusions 

Even if we accept the report’s conclusion 
that private schools do outperform public 
ones, it is still not clear which policy re- 
forms should be implemented. 

For example, a market with more product 
differentiation might simply mean more 
charter schools. Yet, these schools are fully 
publicly funded and so fall afoul of the re- 
port’s criterion that parents must pay di- 
rectly. Even voucher programs include size- 
able subsidies that families then turn over to 
their private schools. 

Finally, the report does not explore or even 
mention any possible adverse consequences 
from a system of independent private 
schools. One of the reasons for having a 
public education system is that of external- 
ities: the recognition that beyond its private, 
individual benefits, education has influences 



on society, on culture, and on what it means 
to be a citizen. Societies do not solely value 
higher test scores; they also care about so- 
cial cohesion and societal inequalities. The 
report appears to implicitly assume that 
these public benefits of education are unim- 
portant or unaffected by the types of schools 
children attend. 

VII. Usefulness of the Report 
for Guidance of Policy 
and Practice 

The report claims that its findings are of 
“profound” importance for U.S. education 
policy. At best this is an overstatement, for 
several reasons. 

First, there is now a substantial evidence 
base on market reforms in the U.S. There 
have been many small-scale reforms, often 
evaluated using high-quality research meth- 
ods. These evaluations are included here. 
But there have also been large-scale re- 
forms, such as charter schools and home- 
schooling, which arguably have radically 
changed the opportunities for parents want- 
ing to choose a school outside of their public 
neighborhood school. Yet, the large body of 
literature on charter schools, much of which 
is high-quality and published in peer- 
reviewed journals, is not mentioned here. 

Accordingly, and contrary to the basic asser- 
tion in the Cato report, there is little warrant 
for U.S. policymakers to draw policy con- 
clusions from tallying the results of the body 
of very uneven international evidence. The 
large and growing body of U.S. evidence 
about school choice and marketization is 
more accessible, applicable and useful than 
figuring out how international evidence ap- 
plies to the U.S. 

Of course, it is hard to summarize all of the 
U.S. literature into a single conclusion. But 
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as explained by Rouse and Barrow (quoted 
above), “The best research to date finds rela- 
tively small achievement gains for students 

offered education vouchers, most of which 

12 

are not statistically different from zero.” 

The argument that the report’s findings have 
profound implications for U.S. policy is 
unconvincing for a second reason: some of 
the report’s international evidence is from 
countries with education systems that are 
dramatically different from the American 
system. Aside from the five studies from the 
U.S., more than half of the 21 studies in- 
cluded in the second (reduced) sample are 
from Pakistan, India, Tanzania, Ghana, and 
Nigeria. It is far from obvious what U.S. 
policymakers might infer from studies in 
these very different countries. This is not a 
matter of one system being preferred, it is 
simply a matter of difference. These five 
countries do not have universal secondary 
schooling (or, at least in the case of Paki- 
stan, even universal elementary schooling), 
have public schooling often tied to religion, 
and have formal labor markets that cover 
only a small subset of the population. In- 
deed, in making any international compari- 
son, scholars need to pay attention to the 
substantial differences in such areas as cur- 
riculum, assessment, funding systems, po- 
litical decentralization, religiosity, wealth, 
teacher labor markets, and, perhaps most 
importantly, the incentives to attend school. 
It is of course possible to learn from the ex- 
periences of other countries, but the lessons 
become harder as countries are less and less 
similar. 

Policymakers and practitioners rightly prefer 
localized evidence: the context of a reform 
matters. Further, education reforms often 



have diverse consequences that need to be 
accounted for. In rural areas, for example, 
private schools may not be able to operate at 
sufficient scale to maximize profit or break 
even. 

Finally, policymakers need to know not only 
whether a reform improves educational out- 
comes but also whether the costs of the re- 
form outweigh the benefits. The Cato report 
does not identify the size of any private 
school advantage, so it is not possible to 
assess the level resources that would need be 
spent to yield such an advantage. The report 
includes 26 tests that purport to assess effi- 
ciency; these ideally should provide an im- 
mediate economic answer. However, many 
of these 26 tests are far from compelling. 
This is so for several reasons, but the pri- 
mary one is that very few studies accurately 
measure costs in public and private schools. 
As well, these studies almost never consider 
the costs of implementing a reform. Conclu- 
sions about efficiency from this evidence 
base should therefore be made very cau- 
tiously. 

In summary, it is possible that private 
schools are superior to public schools when 
all the international evidence is counted. We 
don’t know, and this report does little if any- 
thing to help answer that question. What we 
do know is that the best studies in the U.S. 
and abroad control for many factors before 
drawing any conclusion, and, when these 
factors are accounted for, what is most sur- 
prising is how small the private school ad- 
vantage is — if it even exists. As such, ex- 
panding market forces is unlikely to yield 
dramatic improvements in the quality of the 
U.S. education system. 
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